Image block classification

ABSTRACT

An image processing method involves determining a global motion between a reference frame and a current frame in a frame sequence. A reference block is identified in the reference frame for a current block based on the global motion. A classification parameter is calculated based on the pixel values of the current block and the reference block. The parameter can be used for classifying the block as belonging to the background or foreground of the current frame. The parameter is preferably also utilized in frame rate-up conversion when extrapolating or interpolating new frames.

This application claims the benefit of U.S. Provisional Application No.60/897,512, filed Jan. 26, 2007, the disclosure of which is fullyincorporated herein by reference.

TECHNICAL FIELD

The present invention generally relates to image processing, and inparticular to providing a classification of image blocks in an image ofa video sequence of image frames.

BACKGROUND

The motivation of frame rate up-conversion is that a video sequence withhigher frame rate is generally considered to give higher qualityexperience than a video sequence with lower frame rate. The frame rateof a video sequence can be increased by inserting predicted frames inbetween existing frames. A good approach is to predict the in-betweenframe using bi-directional block based motion estimation [1], searchingfor linear motions between the previous frame and the next frame in theinput video sequence. It is possible to use non-linear approaches thatcan represent acceleration, but the linear approach is used because ofits simplicity and low complexity. The in-between frame is divided intoblocks and to each of these a motion vector must be assigned in someway. However, it is possible that a set of motion vectors are candidatesfor the current block, and only one of these candidate motion vectorswill be selected to represent the motion.

When an evaluation is performed concerning which of these candidatevectors to choose, there are different parameters that can be used. Themost common is the sum of absolute differences (SAD) [2], which iscalculated from the differences of the values between the pixels pointedout in the previous frame and the pixels pointed out in the next frame.

In some cases two (or more) different motion vectors will have low SADvalue but only one can be selected to represent the motion. FIG. 8 showstwo frames 10, 20 in a video sequence. It is not hard (for a human) toguess the contents of the in-between frame 30.

Since the background is unchanged it would probably be the same as inthe two adjacent frames 10, 20, and since there is a ball to the left inthe preceding frame 20 and a ball to the right in the following frame10, it is intuitional to think that there is a ball in the middle of thein-between frame 30. The motion vector that points from the ball inframe 10 to the ball in frame 20 gives a low SAD value and passes themiddle of the image at half of its length.

However, the area in the middle looks exactly, or almost exactly, thesame in the preceding frame 20 as in the following frame 10. A localconsideration would yield that this area will look the same in thein-between frame 30, and the zero motion vector gives a very low SADvalue. Thus, the prior art techniques for selection motion vector mayrun into problems in this situation. Since the zero motion of thebackground is the easiest to capture exactly, this motion vector will bechosen instead of the motion vector that corresponds to the motion ofthe ball. This could result in an in-between picture 30 with no ball atall.

SUMMARY

The present invention overcomes these and other drawbacks of the priorart arrangements.

It is a general object of the present invention to provide aclassification parameter that can be utilized for discriminating betweenforeground and background in an image frame.

It is a particular object of the invention to provide a frameinterpolation/extrapolation that utilizes classification parameters whenselecting candidate pixel blocks.

These and other objects are met by the invention as defined by theaccompanying patent claims.

Briefly, the present invention involves determining a classificationparameter for a current group of at least one image element in a currentframe of a frame sequence. A representation of a global motion of imageelement property values from a reference frame in the sequence to thecurrent frame is determined, preferably based on displacement vectorsassigned or estimated for the groups in the current block. A referencegroup is identified in the reference frame for the current grouputilizing the determined global motion representation. A classificationparameter of the invention is then calculated based on a difference inproperty values of the image elements in the current group and thereference group, preferably as a sum of the absolute values of thedifferences or a sum of the squared differences in property values forimage elements occupying corresponding pixel positions in the currentgroup and the reference group.

The determined classification parameter is indicative of whether thecurrent group belongs to the foreground or the background of the currentframe and can be utilized in a group classification. Then, if theparameter exceeds a threshold, the group is regarded as a foregroundgroup in the frame, otherwise it belongs to the background.

A preferred implementation identifies a second reference group in thereference frame for the current group utilizing the displacement vectorassociated with the current group. The global motion representation isthereafter utilized for identifying a second group in the current framestarting from the identified second reference group. A secondclassification parameter is calculated based on the property values ofthe image elements in the second group and the second reference group.The classification of the current group is then performed based on boththe first and second classification parameters.

The classification parameters of the invention are advantageouslyutilized during frame rate up-conversion when interpolating orextrapolating a new frame in a video sequence based on an existing firstand second frame. A set of multiple candidate groups present in thefirst frame is provided. A classification parameter of the invention isdetermined for each of the candidate groups. Each candidate group has anassociated displacement vector that is utilized for identifying arespective reference group in the second frame. A respective differenceparameter is calculated based on a difference in property values for acandidate group and its identified reference group. The selection ofcandidate group to use when interpolating/extrapolating an image elementgroup in the new frame is performed based on both the prior artdifference measures and the classification parameters of the invention.This selection utilizes the classification parameters for weighting upgroups belonging to the foreground over background groups in theselection. The property values of the group in the new frame aredetermined based on the selected group and its identified referencegroup.

SHORT DESCRIPTION OF THE DRAWINGS

The invention together with further objects and advantages thereof, maybest be understood by making reference to the following descriptiontaken together with the accompanying drawings, in which:

FIG. 1 is a flow diagram of an image processing method according to anembodiment of the present invention;

FIG. 2 is a schematic overview of a video sequence of frames to whichthe teachings of the invention can be applied;

FIG. 3 is an illustration of two frames of a video sequence;

FIG. 4 is a flow diagram illustrating an embodiment of the global motionvector determining step of FIG. 1;

FIG. 5 is a flow diagram illustrating an embodiment of the measurecalculating step of FIG. 1;

FIG. 6 is a flow diagram of an additional step of the method in FIG. 1;

FIG. 7 is a flow diagram of additional steps of the method in FIG. 1;

FIG. 8 is a flow diagram illustrating two image frames with anintermediate image frame to be interpolated;

FIG. 9 is a diagram illustrating a method of selecting motion vectoraccording to an embodiment of the present invention;

FIG. 10 is a flow diagram illustrating two image frames with anintermediate image frame to be interpolated;

FIG. 11 is a diagram illustrating problems in selecting motion vectors;

FIG. 12 is a flow diagram of an interpolation method according to anembodiment of the present invention;

FIG. 13 is a diagram illustrating candidate motion vectors that can beused in the interpolation method;

FIG. 14 is a schematic block diagram of an image processor according tothe present invention;

FIG. 15 is a schematic block diagram of the parameter calculator in FIG.14; and

FIG. 16 is a schematic block diagram of a block interpolator accordingto the present invention.

DETAILED DESCRIPTION

Throughout the drawings, the same reference characters will be used forcorresponding or similar elements.

The present invention generally relates to image and frame processing ina sequence of frames for the purpose of identifying frame portionsbelonging to the background of the frame and the foreground of theframe, respectively.

This classification of frame portions is of high value during frame rateup-conversion to thereby be used in the decision of candidate pixelblocks employed for interpolating or extrapolating pixel values in aconstructed frame in the sequence.

In the present invention, a video or frame sequence comprises multiple,i.e. at least two, frames or pictures. Such a frame can in turn beregarded as composed of a series of one or more slices, where such aslice consists of one or more macroblocks of image elements or pixels.In the present invention, the expression “image element” is used todenote a smallest element of a frame or picture in a sequence. Such animage element has associated image element properties, such as color (inthe red, green, blue, RGB, space) or luminance (Y) and chrominance (Cr,Cb or sometimes denoted U, V). A typical example of an image element isa pixel of a frame or picture. The present invention is particularlyadapted to a video sequence comprising multiple consecutive frames at agiven frame rate. However, the invention is also applicable to a framesequence comprising at least two frames or images. Such a sequence couldcontain two pictures taken at a same time but from different positionsor angles. Thus, the invention is applicable to any frame sequencecomprising at least two frames that could be associated with differenttime instances of a video sequence and/or different positions or angles.

The image elements are organized into groups of image elements. Theexpression “group of image element” denotes any of the prior art knownpartitions of frames and slices into collections of image elements thatare handled together during decoding and encoding. Generally, such agroup is a rectangular (M×N) or square (M×M) group of image elements. Anexample of such a grouping is a macroblock in the video compressionstandard. Such a macroblock generally has a size of 16×16 imageelements. A macroblock can consists of multiple so-called sub-macroblockpartitions, such as 16×8, 8×16, 8×8, 8×4, 4×8 and 4×4 image elements.The 8×8 sub-macroblock partition is often denoted as a sub-macroblock orsub-block, whereas a 4×4 partition is often denoted block.

FIG. 1 is a flow diagram of an image processing method applicable to aframe sequence comprising multiple frames. Each frame in the sequencecomprises multiple groups or blocks of at least one image element,typically multiple, such as 4×4 image elements. The method starts instep S1, which determines a representation of a global motion of imageelement property values from at least a reference portion of a referenceframe to at least a portion of a current frame in the frame/videosequence. This global motion representation is indicative of the globalor overall movement of pixels when going from the reference frame to thecurrent frame in the sequence.

A next step S2 uses the determined global motion representation foridentifying, for a current group of at least one image element in thecurrent frame, a reference group of at least one image element in thereference frame. In other words, a reference group in the referenceframe, to which the global motion representation points when applied tothe position of the current group in the current frame is identified instep S2.

The global motion representation must not necessarily be an integermotion. In clear contrast, property values of a reference group can befiltered to obtain sub-pel values, such as half-pel, quarter-pel or eveneighth-pel values. This means that also decimal number are possible forthe vector components of the global motion representation.

Step S3 calculates a classification parameter for the current groupbased on the property values of the image elements of the current groupand the reference group. This classification parameter is, thus,representative of the difference in image element property valuesbetween the current group and its corresponding reference groupidentified based on the global motion representation. The classificationparameter is furthermore indicative of whether the current group belongsto a foreground or a background of the current frame. Generally anddepending on which particular parameter format selected for theclassification parameter, a large parameter value is obtained if thecurrent group belongs to the foreground, while a comparatively lowerparameter value signals that the current group forms part of thebackground portion of the current frame.

The classification parameter can then be used, as is described furtherherein, for classifying different groups of the current frame but alsofor other purposes such as when interpolating/extrapolating new framesduring frame rate up-conversion.

The operations conducted in FIG. 1 are preferably applied to multiple,such as all, image element groups in the current frame. In such a case,steps S2 and S3 are repeated for all those groups in the current frameto thereby obtain a respective classification parameter for thedifferent image element groups. The method then ends.

FIG. 2 is a schematic overview of a video sequence 1 of frames 10, 20,to which the teachings of the present invention can be applied. Theoriginal video sequence comprises a number of frames 10, 20 associatedwith different time instances, t_(i−1), t₁₊₁, t_(i+3) and so on. In thecase of rate-up conversion, one or more additional frames 30 aredetermined to correspond to a time instance, t_(i), occurring betweentwo original frames 10, 20 in the sequence 1. When applying the presentimage processing method to such a sequence, the frame 10 at timeinstance t_(i+1) can be regarded as the current frame, while theprevious original frame 20 at time instance t_(i−1) will be thereference frame. Alternatively, the different frames 10, 20 canoriginate from different camera views but be of a same or near same timepoint.

FIG. 3 illustrates the current frame 10 and reference frame 20 in moredetail. A current group 12 of at least one image element 14 in thecurrent frame 10 is to be processed according to the present inventionfor the purpose of calculating a classification parameter. A globalmotion representation 50 has been calculated and is indicated in thefigure. This global motion representation 50 can be regarded as a vectorthat points from the corresponding position of the current group 12 butin the reference frame 20 up to a group 28 of at least one image element24 that will be the reference group 28 in the reference frame 20 for thecurrent group 12. As is shown in the figure, the size of the referencegroup 28 in terms of the number of image elements 24 could the same asthe corresponding size of the current group 12. However, if sub-pixelvalues are used, these pixel values can be determined by filtering alarger portion of the reference frame than the group size, which is wellknown in the art.

The global motion representation 50 of the present invention can takeany vector value

${v = \begin{bmatrix}x \\y\end{bmatrix}},$ranging from the zero vector up to non-zero values for the vectorcomponents x and y, depending on how the pixel parameter values aremoved when going from the reference frame 20 to the current frame 10 inthe sequence.

FIG. 4 is a flow diagram illustrating a preferred embodiment ofdetermining the global motion representation of the present invention.The method starts in step S10, where a vector set is provided. Thisvector set comprises, for each image element group in at least a portionof the current frame a respective associated displacement or motionvector referring to a reference group of at least one image element inthe reference frame. Thus, each group in at least a portion of thecurrent frame, preferably each group in the frame, has an assigneddisplacement vector that is pointing to or associated with a referencegroup in the reference frame.

The displacement vectors can be provided from a coded motion vectorfield of a video codec, such as H.264. Such motion vectors aretraditionally used in inter coding of frames and can be re-used but foranother purpose according to the invention. If no such motion vectorsare available from the video codec, they can be determined from a motionestimation search. In such a case, a dedicated motion estimation searchis conducted, preferably according to prior art algorithms but for thepurpose of generating a motion vector set that can be used fordetermining the global motion representation of the invention.

Generally, each image element group in the current frame can have anassociated motion vector generated by the video codec or from the motionestimation. However, some of the groups may not have an assigned motionvector as these are coded as intra blocks by the video codec. In such acase, such groups can be omitted from the processing of the motionvectors of the invention. This means that only a portion (though a majorportion) of the groups in the current frame and their assignedmotion/displacement vectors are utilized in the following step S11 forcalculating the global motion representation.

The next step S11 uses the displacement vectors from the provided(fetched or calculated) vector set from step S10 to determine a globalmotion vector. In a simple implementation, the global motionrepresentation is determined as an average vector of the displacementvectors in the vector set. This is a computationally simple embodiment,though far from optimal for the purpose of obtaining an accurate globalmotion representation. Therefore, in a preferred embodiment of step S11,a position-dependent global motion vector or representation havingvector component values that can vary for different image elementpositions in the current frame, i.e. v=v(x,y), is determined in stepS11.

A preferred implementation of step S11 utilizes the followingrepresentation of the global motion representation:v=Ax+bwhere

$x = \begin{bmatrix}x \\y\end{bmatrix}$is the position of a current group in the current frame,

$v = \begin{bmatrix}v_{x} \\v_{y}\end{bmatrix}$is the global motion representation of the current group,

$A = \begin{bmatrix}a_{11} & a_{12} \\a_{21} & a_{22}\end{bmatrix}$and

$b = \begin{bmatrix}b_{1} \\b_{2}\end{bmatrix}$are a matrix and a vector that are to be estimated based on thedisplacement vectors provided in step S10. In order to calculate thevalues for the matrix A and the vector b, a least square method ispreferably used for the provided displacement vector. Thus, the matrixand vector that gives a best result, in terms of minimizing a squareddifference between the displacement vectors and the global motionrepresentation, are estimated in step S11. The final global motionrepresentation v=Ax+b captures most common background motions, such ascamera panning, zooming and rotation.

The above concept can of course be applied to other parameterizations ofa global motion representation, such as

$v = {{\begin{bmatrix}a_{11} & a_{12} \\a_{21} & a_{22}\end{bmatrix}\begin{bmatrix}x^{2} \\y^{2}\end{bmatrix}} + {\begin{bmatrix}b_{11} & b_{12} \\b_{21} & b_{22}\end{bmatrix}\begin{bmatrix}x \\y\end{bmatrix}} + \begin{bmatrix}c_{1} \\c_{2}\end{bmatrix}}$or higher order components. The method then continues to step S2 of FIG.1, where the determined representation of the global motion is applied,using the group coordinates x and y of the current group to calculatethe global motion at that point and identify the reference group in thereference frame.

The usage of displacement vectors from the video codec or from adedicated motion estimation search is a particular embodiment ofobtaining a displacement vector set that are used for determining aglobal motion representation of the present invention. Other embodimentscan instead be used and are contemplated by the invention. For instance,a motion estimation that is based on phase correlation can be used toobtain a representation of the global motion. Another example is ofmotion estimation for the global motion is pel-recursive, i.e.pixel-based motion estimation.

FIG. 5 is a flow diagram illustrating a particular embodiment of theparameter calculating step S3 of FIG. 1. The method continues from stepS2 of FIG. 1. A next step S20 calculates a difference between theparameter values of the image elements in the current group and in theidentified reference group. These differences are calculated for eachimage element in the current group and based on the property value ofthat image element and the property value of the reference image elementhaving the corresponding position in the reference group as the imageelement has in the current group. In other words, the differences arecalculates as D_(x,y)=IE_(x,y)−RIE_(x,y), where IE_(x,y) is the propertyvalue of image element at position x,y in the current group, RIE_(x,y)is the property value of the reference image element at position x,y inthe reference group and D_(x,y) is the difference.

A next step S21 calculates the classification parameter based on theabsolute values of the differences. Different embodiments can be used inthis parameter calculation based on the absolute values. A firstembodiment utilizes a classification parameter that is based on the sumof the absolute differences (SAD):

${CP} = {\sum\limits_{y}{\sum\limits_{x}{{{IE}_{x,y} - {RIE}_{x,y}}}}}$

Another preferred embodiment of classification parameter is the sum ofsquared differences (SSD):

${CP} = {\sum\limits_{y}{\sum\limits_{x}\left( {{IE}_{x,y} - {RIE}_{x,y}} \right)^{2}}}$

In both these cases, a high SAD or SSD values is obtained for groupsbelonging to the foreground of the current frame, while backgroundgroups have comparatively lower SAD and SSD values.

FIG. 6 is a flow diagram illustrating an additional step of the imageprocessing method of FIG. 1. The method continues from step S3 inFIG. 1. A next step S30 classifies a current group as belonging to thebackground or the foreground of the current frame based on thecalculated classification parameter. The classification of step S30 is,as will be described further herein, preferably performed based on acomparison of the classification parameter and a threshold value. Insuch a case, the current group is classified as belonging to theforeground of the current frame if the calculated classificationparameter exceeds the parameter and is classified as belonging to thebackground if it is smaller than the threshold.

With reference to FIGS. 3 and 7, preferred additional steps of the imageprocessing method of FIG. 1 are illustrated. The method continues fromstep S3 in FIG. 3. A next step S40 identifies a second reference group22 of at least one image element 24 in the reference frame 20 for thecurrent group 12 in the current frame 10. This identification isperformed based on the current group 12 and a displacement vector orrepresentation 16 associated with the current group 12. The displacementvector 16 points from a corresponding position the current group 12would have had in the reference frame 20 and up to the second referencegroup 22.

The displacement vector 16 can be fetched from the vector set previouslyused for calculating the global motion representation 50 for the currentframe 10. Thus, the vector 16 can be obtained from the video codec or bedetermined in a motion estimation search.

A next step S41 calculates a so-called discard parameter based on theproperty values of the image elements 24 in the second reference group22 and the property values of the image elements 14 in the current group12. This discard parameter is preferably based on the absolute values ofthe pairwise differences in property values for image elements occupyingcorresponding positions in the current group 12 and the reference group22. The calculation is preferably performed in a similar manner to theclassification parameter calculating described above with the differencethat now it is the image property values of the second reference group22 identified by the displacement vector 16 that are used and not theproperty values of the first reference group 28 identified based on theglobal motion representation 50.

The discard parameter is preferably of a same parameter type as thepreviously described first classification parameter. In other words, ifthe classification parameter is a SAD (SSD) value, the discard parameteris preferably also a SAD (SSD) value.

The calculated discard parameter DP is compared to a first thresholdvalue T₁ in step S42. If the discard parameter is smaller than the firstthreshold, the method continues to step S43 otherwise the method endsand no classification of the current group is possible or theclassification is solely based on the first calculation parametercalculated in step S3 of FIG. 1.

In step S43, a second group 18 of at least one image element 14 isidentified in the current frame 10 based on the second reference group22 in the reference frame 20 and the global motion representation 50.This second group 18 is identified by applying the global motionrepresentation 50 from a corresponding position the second referencegroup 22 would have had in the current frame 10 and then therepresentation 50 points to the position of the second group 18. As isevident from FIG. 3, the direction of the global motion representation50 is opposite when going from the current group 12 to the firstreference group 28 as compared to starting from the second referencegroup 22 and ending at the second group 18. Thus, if the global motionrepresentation 50 had the values

$v = \begin{bmatrix}v_{x} \\v_{y}\end{bmatrix}$in the former case, it preferably has the value

${- v} = \begin{bmatrix}{- v_{x}} \\{- v_{y}}\end{bmatrix}$in the second case.

A next step S44 calculates a second classification parameter based onthe differences in image element property values of the second referencegroup 22 and the second group 18. This step S44 is preferably performedin a similar way to the calculation of the first classificationparameter with the differences in that it is the property values fromthe second group 18 and second reference group 22 that are used and notthe values from the current group 14 and the first reference group 28.Thus, pairwise differences between property values of image elements 14,24 occupying corresponding positions in the second group 18 and thesecond reference group 22 are calculated. The absolute values of thedifferences are then summed to get a SAD-based second classificationparameter or the squared differences are summed to obtain the secondclassification parameter in the SSD form. The second classificationparameter CP₂ is, though, preferably in the same parameter format as thefirst classification parameter CP₁ and the discard parameter.

The first classification parameter CP₁ is preferably compared to asecond threshold value T₂ in step S45. If the first classificationparameter is below the second threshold, the method continues to stepS46 where the second classification parameter CP₂ is compared to a thirdthreshold value T₃. In a typical implementation, the third thresholdvalue is equal to the second threshold value. The threshold values couldbe fixed or adapted or determined based on the particular videosequence.

If the second classification parameter is below the third threshold, themethod continues to step S47, where the current group 12 is classifiedas belonging to the background of the current frame 10. However, if thesecond classification parameter would exceed the threshold in step S46,the method continues to step S48. In this case, the first parameter isbelow its threshold value, while the second classification parameterexceeds its threshold value. In such a case, the classification of thecurrent group 12 could be regarded as indefinite or alternatively onlythe first classification parameter is utilized, i.e. the group 12 willbe regarded as a background group.

If the first classification instead exceeds the second threshold valuein step S45, the method continues to step S49, where the secondclassification parameter is compared to the third threshold insimilarity to step S46. Once again the classification of the currentgroup 12 could be regarded as indefinite if the second classificationparameter is below the third threshold, i.e. one parameter exceeds itsthreshold while the other falls below. However, if also the secondclassification parameter exceeds its compared threshold in step S49, themethod continues to step S50, where the current group 12 is classifiedas belonging to the foreground of the current frame 10.

An alternative that avoids an indefinite classification is to calculatea weighted sum of the two classification parameters, preferably usingsame parameter weights. The sum is then compared to a classificationparameter. Another possibility is to classify a current group asbelonging to the foreground if and only if both classificationparameters exceed the respective threshold values, otherwise the groupis regarded as belonging to the background.

It is anticipated by the present invention that the order of comparisonscan be exchanged so that the second classification parameter is firstcompared with the third threshold before the first parameter is comparedto the second threshold value. Comparatively, the comparison with thediscard parameter can be performed or after the comparisons of theclassification parameters.

Thus, in this particular embodiment, the classification of the currentgroup 12 is performed based both on the first classification parameterand the second classification parameter and preferably also the discardparameter. However, in another embodiment of the present invention thesteps S41 and S42 are omitted, so then the classification is madewithout the discard parameter and instead utilizes the first and secondparameters.

The method described above is preferably performed on all groups 12 inthe current frame 10 to thereby form a complete classification of thegroups as background groups or foreground groups (or indefiniteclassification). In an optional continuation of the image processingmethod, a refined updating of the global motion representation can beperformed based only on those groups classified as belonging to thebackground of the current frame 10. In such a case, the determinationcan be performed according to any of the previously describedembodiments, though only utilizing the background groups. For instance anew estimation of the matrix A′ and vector b′ can be performed utilizingthe displacement vectors associated with the background groups todetermine an updated global motion representation v′.

A new classification parameter calculation and classification parameterprocess can then be performed based on the updated global motionrepresentation. This can be repeated in an iterative way by generatingan updated global motion representation based on a previously performedcalculation to get a more and more accurate global motion representationand group classification. However, for the majority of image frames theglobal motion determination and the classification need only beperformed once, i.e. no updating or refinement.

The classification parameter determined according to the presentinvention and the classification of groups into background versusforeground groups can advantageously be utilized when interpolating orextrapolating new frames in connection with frame rate up-conversion.

FIG. 8 illustrates a situation that can arise during such anup-conversion. The image to the left corresponds to a previous referenceframe 20 in a video sequence and illustrates a portion of a footballfield with a football to the left in the picture. A later frame 10 inthe video sequence basically shows the same football field portion butnow the football is to the right of the picture. In the rateup-conversion an intermediate frame 30 is to be interpolated between thereference frame 20 and the current frame 10. For the human viewer it isnot hard to guess that this intermediate frame 30 would show thefootball field with the football basically in the middle of the lowerhalf of the picture. However, the prior art rate up-conversion schemesthat only performs the interpolation based on motion vectors betweengroups in the current 10 and reference 20 frames and do not base theinterpolation at least partly based on a foreground/backgroundclassification will probably generate an intermediate frame 30 with nofootball at all as described in the background section.

FIG. 9 schematically illustrates the superior interpolation of thepresent invention for the example illustrated in FIG. 8. In this case,the global motion of image element property values from the previousframe 20 at time instance t_(i−1) to the current frame 10 at timeinstance t_(i+1) will be the zero motion (since the background does notmove at all between the two frames 10, 20). There are two candidatedisplacement vectors for the middle area in the intermediate frame 30 attime instance t_(i): the one that captures the motion of the ball,represented by displacement vector 40, and the one that captures the(zero) motion of the background in the middle, represented by the vector42. If the global motion representation 50 is applied to an imageelement group covering the ball in the previous frame 20, we arrive atan image element group in the subsequent frame 10 covering grass.Correspondingly, if we start at an image element group in the lowerright of the previous frame 20, i.e. containing grass of the background,and moves according to the global motion into the subsequent frame 10,we would then arrive at a group covering the football. This means thatthe area pointed out by the displacement vector 40 associated with themotion of the ball has a bad match in the global motion 50 since one ofthe frames 10, 20 contains a ball and the other frame 10, 20 containsgrass at those specific areas. Thus, a classification parameter that iscalculated based on the pixel parameter values of grass group and thefootball group would have a high parameter value.

However, the (zero) displacement vector 42 from the grass in the middleof the lower half of the previous frame 20 to a grass group in themiddle of the lower half of the subsequent frame 10 have a good matchwith the global motion 50 since the displacement vector 42 and theglobal motion representation 50 are both zero in this case. Therefore, aclassification parameter calculated based on the difference in propertyvalues of the groups in the previous 20 and subsequent frames 10identified by applying the global motion 50 would in this case below asboth shows background grass.

Applying the image processing of the present invention to this simpleexample, the image element groups corresponding to the football would beclassified as foreground, while remaining groups are background. Thisclassification is then utilized when selecting between the two candidatedisplacement vectors 40, 42. Therefore, by basing the interpolation atleast partly on the classification of the invention, the (correct)displacement vector 40 would be selected for the middle area in thelower half of the intermediate frame 30 even though the other (zero)candidate vector 42 would probably achieve a lower discrimination value,such as SAD or SSD value, as compared to the football vector 40.

One might think that it would be easier to compare a candidate motionvector 40, 42 to the global motion 50 directly; if they differ enough,the vector 40, 42 corresponds to a foreground object. Unfortunately sucha process can easily be fooled. In the case with the ball in FIG. 8,there is an area to the right of the football in the previous 20, seeFIG. 10, which resembles an area to the left of the ball in subsequentframe 10. The displacement vector 44 for this third alternative is shownin FIG. 11 together with the other two candidates 40, 42 describedabove.

A motion vector 44 that points between the marked areas in FIG. 10 aredifferent from the (zero) global motion 50, but still does notcorrespond to a foreground object. This is evident as the classificationparameter obtained by calculating the difference in image elementproperty values from a group in the circle in the previous frame 20 anda group in the subsequent frame 10 identified based on the global motionrepresentation 50 is comparatively low as is the parameter obtained bystarting from a group in the circle in the subsequent frame 10 andmoving according to the global motion 50 to a group in the previousframe 20. In this case, all four groups will correspond to grass objectsand therefore form part of the background. Thus, if displacement vectors40, 42, 44 would directly be compared to the global motionrepresentation 50, groups would become misclassified as in this case.However, with the present invention it is easy to see that the areaspointed out by the third alternative is part of the background sincethey both have good matches in the global motion.

FIG. 12 is a flow diagram illustrating a method of estimating propertyvalues of a group of at least one image element in a frame associatedwith a time instance in a video sequence. This estimation is preferablyperformed as a part of a rate up-conversion procedure to add one or moreframes to a video sequence through frame interpolation or extrapolation.

The method starts in the optional step S60, where at least two frames inthe video sequence to use in the estimation are identified. In a typicalembodiment, one of the frames corresponds to a previous time instancerelative the intermediate frame to be interpolated while the other ofthe frames corresponds to a subsequent time instance in the sequence. Ina preferred embodiment, the two frames are the frames that arepositioned immediately before and immediately after the intermediateframe in terms of time. In other words, the frames could be regarded asneighboring frames. In this frame interpolation, more than two framescan actually be used, such as using N previous frames corresponding totime instance t_(i−1), t_(i−3), . . . , t_(i+1−2N) and M followingframes corresponding to time instances t_(i+1), t_(i+3), . . . ,t_(i−1+2M), for interpolating a frame of time instance t_(i).

Correspondingly, when extrapolating a frame at time t_(i) two or moreprevious frames at times t_(i−1), t_(i−3), t_(i+1−2N) or two or morefollowing frames at times t_(i+1), t_(i+3), . . . , t_(i−1+2M) are used.

A similar procedure regarding interpolation and extrapolation can beapplied in the spatial domain instead of the time domain. In such acase, a new “view” is interpolated or extrapolated from two images orframes with different angles and/or displacements though, possibly, sametime.

A next step S61 provides a set of multiple, i.e. at least two, candidategroups in a first (previous or subsequent) frame associated with aprevious or following time instance in the video sequence compared tothe frame to be determined. Each of these candidate groups comprises atleast one image element and is associated with a respective displacementrepresentation or vector. These displacement vectors can be fetched fromthe inter coding of the frame, i.e. from the video codec, or they can bedetermined from a motion estimation search. A next step S62 determines aclassification parameter for each of the candidate groups. Thisparameter determination is performed according to any of the previouslydescribed embodiments of the invention.

A next step S63 identifies, for each candidate group, a respectivereference group of at least one image element in a second (previous orsubsequent) frame associated with a previous or following time instancein the video sequence. The reference group associated with the candidategroup is preferably identified based on the displacement vectorassociated with the candidate group. Thus, the displacement vectorpoints, when applied from a corresponding position that the candidategroup would have had in the second frame, towards the reference group.Each candidate group now has a determined classification parameter andan identified reference group.

The following step S64 calculates, for each candidate group, adifference measure representative of a difference in property values ofthe image elements in the candidate group and its identified referencegroup. In a preferred embodiment, the measure is calculated based on theabsolute values of the difference in property values for image elementsoccupying corresponding positions in the candidate group and thereference group. Preferred examples of such difference measured includeSAD and SSD.

A next step S65 selects a candidate group from the provided group setbased on the calculated difference measures and the determinedclassification parameters. The selection of step S65 is preferablyperformed based on a selection parameter calculated from theclassification parameter and the difference measure, such a weighted sumthereof. A preferred such selection parameter e could be defined ase=w₁×DM−w₂×CP, where DM is the difference measure and CP is theclassification parameter for a candidate group and w₁, w₂ are weights.The candidate group having the smallest associated value e is thenselected in step S65. By investigating the selection parameter e, onerealizes that a group being classified as foreground (highclassification parameter) is favored over a background group (lowclassification parameter). Furthermore, candidate groups having anassociated reference group that is a good match to the image elementproperty values of the candidate group (low difference measure) islikewise favored as compared to candidate groups having worse matches inthe second frame. In a preferred embodiment, w₁>w₂ to put a largerweight on the influence of the difference measure in the selection thanthe group classification. It is evident from the discussion above thatthe weights are non-zero (positive) weights.

Finally step S66 determines the property values of the group in theinterpolated/extrapolated frame based on the property values of theselected candidate group and the reference group associated with theselected candidate group. In a preferred embodiment, the image elementproperty values are determined as a linear combination of the propertyvalues of the selected candidate group and the associated referencegroup. The weight applied to the property values in the selected groupand the weight of the property values in the associated group arepreferably determined based on difference in time between theinterpolated/extrapolated frame and the first frame with the selectedcandidate group and the interpolated/extrapolated frame and the secondframe, respectively. In other words, larger weights are used if thedistance in time is small as compared to longer time distances. Thevalues of the frame weights can also be utilized to reflect accelerationas is known in the art.

In the embodiment described above, the reference groups associated withthe candidate groups are identified based on the displacement vectorsassigned to or estimated for the candidate groups. This then presumesthat a same vector is used for traveling from a candidate group to thegroup to be interpolated/extrapolated as when going from the group to bedetermined to a reference group. The present invention is though notlimited thereto.

In another embodiment, a second set of multiple second candidate groupspresent in the second frame is provided together with the provision ofthe first set of multiple first candidate groups in the first frame.Also these second candidate groups comprise at least one image elementeach and preferably have a respective displacement vector. A secondclassification parameter is determined as previously described for eachof the second candidate groups in addition to the first classificationparameters for the first candidate groups. A difference measure can thenbe calculated for each pair of one first candidate group and one secondcandidate group from the first and second sets, respectively.Alternatively, not all combinations of first and second candidates aretested but only a limited portion thereof, reflecting plausiblecombinations of candidate groups, such as groups present on the sameframe positions in the first and second frames and groups havingassociated displacement vectors identifying other candidate groups inthe other frames.

A first and a second candidate group are thereafter selected based onthe first and second classification parameters and the differencemeasures. The image element property values of the group to bedetermined are calculated based on the property values of these selectedcandidate groups as previously described.

FIG. 13 illustrates a portion of a video sequence 1 having a first frame10 and a second frame 20 and an intermediate frame 30 to be determinedduring frame rate-up conversion. A group 32 of image elements 34 to bedetermined is indicated in the intermediate frame 30. Suitable firstcandidate groups 11, 13, 15 are shown in the first frame 10 andcorresponding second candidate groups 21, 23, 25 are shown in the secondframe 20. These candidate groups typically comprise the groups 11, 21having the corresponding position in the first 10 and second 20 frame asthe group 32 has in the intermediate frame 30. The displacement vectors42, 62 of these groups 11, 21 have been indicated in the figure, passingthrough the group 32 to be determined. Other candidates are obtained byutilizing the displacement vectors from the neighboring groups 13, 23 ofthese candidate groups 11, 21. Also those groups 15, 25 havingassociated displacement vectors 40, 60 that pass through the group 32 inthe intermediate frame 30 are preferred candidate groups according tothe invention.

The group classification and the classification parameters calculatedaccording to the present invention can, as has been disclosed in theforegoing, advantageously be utilized during frame rate up-conversionfor interpolating or extrapolating new frames of image element groups.The present invention, though, has other applications in the field ofimage processing. For instance, the classification can be utilized inconnection with an error concealment mechanism for lost frames, where adistorted frame or a part thereof is replaced by unidirectional(extrapolation) or bi-directional (interpolation) prediction fromneighboring frames in a video sequence. The refinement achieved byutilizing the classification parameter in such frame (part) replacementmay lead to more accurate replacement as compared to prior artsolutions.

FIG. 14 is a schematic block diagram of an image processor 100 accordingto the present invention. The processor 100 is applicable to a framesequence, such as a video sequence, comprising multiple frames havinggroups of at least one image element. A global motion determiner 120 isarranged in the processor 100 for determining a representation of aglobal motion of image element property values from at least a portionof a reference frame to at least a portion of a current frame in thesequence. The determiner 120 is preferably connected to a set provider110, which is arranged for providing a vector set comprising, for eachimage element group in the portion of the frame, a displacement vectorreferring to a reference group in the reference frame. The set provider110 can fetch this set from an internal or external video codec, orinclude functionality for estimating the displacement vectors in amotion estimation search. The determiner 120 preferably generates theglobal motion representation as one of the previously describedposition-dependent global motion vectors, by determining matrix A andvector b of the global motion representation.

A group identifier 130 is provided in the processor 100 for identifying,for a current group in the current frame, a reference group in thereference frame based on the global motion representation from thedeterminer 120.

A parameter calculator 140 calculates a classification parameter for thecurrent group based on a difference in image element property values ofthe current group and the reference group identified by the groupidentifier 130. This parameter is indicative of whether the groupbelongs to the background or foreground of the current frame. As aconsequence, an optional but preferred group classifier 150 isimplemented for classifying the current group as a background orforeground group using the classification parameter. In thisclassification operation, the classifier 150 preferably compares theparameter with a threshold value and classifies the block as aforeground block if the parameter exceeds the threshold value, otherwiseit is regarded as a background group.

In a preferred embodiment, the group identifier 130 also identifies asecond reference group in the reference frame. In clear contrast to thefirst reference group, which was identified based on the global motionrepresentation determined by the global motion determiner, the secondreference group is identified based on the displacement vectorassociated with the current group and can be fetched or estimated by theset provider 110. The group identifier 130 also utilizes the globalmotion representation and the new identified second reference group foridentifying a second image element group in the current frame. Takentogether we therefore have four groups: the current group in the currentframe; the first reference group in the reference frame identifiablestarting from the current group and utilizing the global motionrepresentation; the second reference group in the reference frameidentifiable starting from the current group and utilizing itsassociated displacement representation; and the second group in thecurrent frame identifiable starting from the second reference group andutilizing the global motion representation.

The parameter calculator 140 calculates a first classification parameterbased on the image element property values for the current group and thefirst reference group and calculates a second classification parameterbased on the property values of the second group and the secondreference group. The group classifier 150 uses both the first and secondclassification parameters in the classification of the current group.Thus, if both the first and second parameter exceeds a respectivethreshold value, which are preferably the same, the current group is aforeground group, and if both parameters are below the respectivethreshold values, the group is a background group. Otherwise theclassification can be regarded as indefinite.

Alternatively, the parameter calculator 140 calculates a singleclassification parameter as a weighted sum of the first and secondclassification parameter. This single parameter is compared to thresholdvalue and used for discriminating between a foreground group (highparameter value) and background group (low parameter value).

In a further embodiment, the calculator 140 also calculates a discardparameter that is utilized by the classifier 150 in the groupclassification. The discard parameter is calculated based on theabsolute values of the differences in property values for image elementshaving corresponding positions in the current group and in the secondreference group. The discard parameter can, for instance, be a SAD orSSD parameter. The classifier 150 preferably compares the discardparameter with an associated threshold value and could abort theclassification or only elect to utilize the first classificationparameter if the discard parameter exceeds its threshold.

The units 110 to 150 of the image processor 100 can be provided inhardware, software and/or a combination of hardware and software. Theunits 110 to 150 can be implemented in a video or frame processingterminal or server, such as implemented in or connected to a node of awired or wireless communications system. Alternatively, the units 110 to150 of the image processor 100 can be arranged in a user terminal, suchas TV decoder, computer, mobile telephone, or other user appliancehaving or being connected to an image rendering device.

FIG. 15 is a schematic block diagram illustrating a possibleimplementation of the parameter calculator 140 of FIG. 14. Thecalculator 140 comprises a difference calculator 142 for calculatingdifferences in property values of image elements occupying correspondingpositions in two different groups, such as the current group and thefirst reference group. A SAD/SSD calculator 144 calculates aclassification parameter based on the absolute values of the differencesfrom the difference calculator 142, such as the sum of the absolutevalues or the sum of the squared absolute values.

The units 142 and 144 of the parameter calculator 140 may be implementedin hardware, software and/or a combination of hardware and software.

FIG. 16 is a schematic block diagram of device 200 for determining agroup of image elements by estimating the property values of the atleast one image element in the group. The device 200 optionallycomprises a frame identifier 210 for identifying at least a first and asecond frame in a video sequence. These two frames are associated withdifferent time instances in the sequence as compared to a current framecomprising the group to be determined. In the case of groupinterpolation, the first frame is a previous or following frame, whilethe second frame is a following or previous frame. For groupextrapolation both the first and second frames are previous or followingframes in relation to the current frame.

A set provider 220 is arranged in the device 200 for providing a set ofmultiple candidate groups in the first frame. Each of the candidategroups comprise at least one image element and has a respectivedisplacement vector. An image processor 100 of the present invention asdescribed above and disclosed in FIG. 14 is arranged in the device 200for calculating a respective classification parameter for each candidategroup in the set.

A group identifier 230 identifies a respective second group in thesecond frame for each of the candidate groups. This identification ispreferably performed based on the displacement vectors associated withthe candidate groups. A measure calculator 240 calculates a differencemeasure for each candidate group, where the measure is representative ofa difference in property values of the candidate group and itsidentified second group. The measure is preferably a SAD or SSD measure,i.e. based on the absolute values of the image element differences.

The device 200 also comprises a group selector 250 for selecting acandidate group from the provided set based on the classificationparameters from the image processor 100 and the difference measures bythe calculator 240. The selector 250 preferably calculates a weighteddifference or sum of the measure and classification parameter for eachcandidate and then selects the candidate group leading to the smallestweighted difference/sum. A value determiner 260 determines the propertyvalues of the current group based on the property values of the selectedcandidate group and its associated second group, typically a linearcombination of the image element values of these two groups.

As has previously been indicated, the set provider 220 can provideanother set of multiple second candidate groups in the second frame. Insuch a case, the image processor 100 also calculates secondclassification parameters for these candidate groups. The selection offirst and second candidate group by the group selector 250 is performedbased on the first and second classification parameters and thedifference measures. The property values of the selected first andsecond candidate group are utilized by the value determiner 260 forinterpolating or extrapolating the property values of the current group.

The units 100, 210 to 260 of the determining device 200 can be providedin hardware, software and/or a combination of hardware and software. Theunits 100, 210 to 260 can be implemented in a video or frame processingterminal or server, such as implemented in or connected to a node of awired or wireless communications system. Alternatively, the units 100,210 to 260 of the determining device 200 can be arranged in a userterminal, such as TV decoder, computer, mobile telephone, or other userappliance having or being connected to an image rendering device.

It will be understood by a person skilled in the art that variousmodifications and changes may be made to the present invention withoutdeparture from the scope thereof, which is defined by the appendedclaims.

REFERENCES

-   [1] Choi, B. T., Lee, S. H., & Ko, S. J., 2000, New frame rate    up-conversion using bi-directional motion estimation, IEEE Trans.    Consum. Electron., Volume 46, Number 3, pp. 603-609-   [2] Zhai, J., Yu, K., Li, J. & Li, S., 2005, A Low Complexity Motion    Compensated Frame Interpolation Method, The 2005 IEEE International    Symposium on Circuits and Systems (ISCAS2005), Kobe, Japan, 23-26    May, 2005

The invention claimed is:
 1. An image processing method applicable to aframe sequence comprising multiple frames, each frame comprisingmultiple groups of image elements, said method comprising the steps of:determining a global motion representation of image element propertyvalues representing a movement of corresponding image elements from aposition in at least a reference portion of a reference frame to adifferent position in at least a portion of a frame; identifying, for afirst group of at least one image element in said frame, a firstreference group of at least one image element in said reference framebased on said global motion representation; calculating a firstclassification parameter as being representative of a difference inimage element property values of said first group and said firstreference group; identifying a second reference group of at least oneimage element in said reference frame based on a displacement vectorassociated with said first group; identifying a second group of at leastone image element in said frame based on said global motionrepresentation and said second reference group; calculating a secondclassification parameter representative of a difference in image elementproperty values of said second reference group and said second group,said first classification parameter and said second classificationparameter being indicative of whether said first group belongs to aforeground or a background of said frame; and providing a vector setcomprising, for each group of image elements in said at least a portionof said frame, a respective displacement vector referring to a referencegroup of image elements in said reference frame of said frame sequence;wherein said determining step comprises determining, based on at least aportion of said displacement vectors of said vector set, said globalmotion representation, wherein a position-dependent global motion vectoris determined based on said at least a portion of said displacementvectors of said vector set, wherein the position-dependent global motionvector is determined by estimating, based on said at least a portion ofsaid displacement vectors of said vector set, elements of matrix$A = \begin{bmatrix}a_{11} & a_{12} \\a_{21} & a_{22}\end{bmatrix}$ and a vector b=[b₁b₂]^(T) by a least square method todetermine said position-dependent global motion vector as having aformula:v=Ax+b where v=[v_(x)v_(x)]^(T) is said global motion vector, v_(x) is afirst vector component of said position-dependent global motion vectorin a first direction, v_(y) is a second vector component of saidposition-dependent global motion vector in a second perpendiculardirection, x=[xy]^(T) is an image element position in said frame.
 2. Themethod according to claim 1, wherein said step of calculating the firstclassification parameter comprises the steps of: calculating, for eachimage element in said first group, a difference between said imageelement property value of said image element and said image elementproperty value of an image element having a corresponding image elementposition in said first reference group; and calculating said firstclassification parameter based on absolute values of said differences.3. The method according to claim 1, further comprising the steps of:calculating, for each image element in said second reference group ofimage elements, a difference between said image element property valueof said image element and said image element property value of an imageelement having a corresponding image element position in said secondgroup; and calculating a discard parameter based on absolute values ofsaid differences.
 4. The method according to claim 1, further comprisingthe step of classifying said first group as belonging to said backgroundor said foreground based on said first classification parameter and saidsecond classification parameter.
 5. The method according to claim 3,further comprising the step of classifying said first group as belongingto a background or a foreground based on said first classificationparameter, said second classification parameter and said discardparameter.
 6. The method according to claim 4 wherein said classifyingstep comprises the step of classifying said first group as belonging tosaid background or said foreground based on a comparison of said firstclassification parameter and a first threshold value and based on acomparison of said second classification parameter and a secondthreshold value.
 7. The method according to claim 6, wherein saidclassifying step comprises the steps of: classifying said first group asbelonging to said foreground if said first classification parameterexceeds said first threshold value and said second classificationparameter exceeds said second threshold value; and classifying saidfirst group as belonging to said background if said first classificationparameter is smaller than said first threshold value and said secondclassification parameter is smaller than said second threshold value. 8.The method according to claim 1, further comprising the steps of:performing said identifying step and said calculating step for eachgroup of image elements in said at least a portion of said frame; anddetermining an updated global motion of said image element propertyvalues from said at least a reference portion of said reference frame tosaid at least a portion of said frame based on said groups in said atleast a portion of said frame classified as belonging to saidbackground.
 9. An image processor applicable to a frame sequencecomprising multiple frames, each frame comprising multiple groups ofimage elements, said processor comprising: a global motion determinerfor determining a global motion representation of image element propertyvalues representing a movement of corresponding image elements from aposition in at least a reference portion of a reference frame to adifferent position in at least a portion of a frame; a group identifierfor i) identifying, for a first group of at least one image element insaid frame, a first reference group of at least one image element insaid reference frame based on said global motion representation, ii)identifying a second reference group of at least one image element insaid reference frame based on a displacement vector associated with saidfirst group, and iii) identifying a second group of at least one imageelement in said frame based on said global motion representation andsaid second reference group; a calculator for i) calculating a firstclassification parameter as being representative of a difference inimage element property values of said first group and said firstreference group and ii) calculating a second classification parameterrepresentative of a difference in image element property values of saidsecond reference group and said second group, said first classificationparameter and said second classification parameter being indicative ofwhether said first group belongs to a foreground or a background of saidframe; and a set provider for providing a vector set comprising, foreach group of image elements in said at least a portion of said frame, arespective displacement vector referring to a reference group of imageelements in said reference frame of said frame sequence; wherein theglobal motion determiner is configured to determine, based on at least aportion of said displacement vectors of said vector set, said globalmotion representation, wherein a position-dependent global motion vectoris determined based on said at least a portion of said displacementvectors of said vector set, wherein the position-dependent global motionvector is determined by estimating, based on said at least a portion ofsaid displacement vectors of said vector set, elements of matrix$A = \begin{bmatrix}a_{11} & a_{12} \\a_{21} & a_{22}\end{bmatrix}$ and a vector b=[b₁b₂]^(T) by a least square method todetermine said position-dependent global motion vector as having aformula:v=Ax+b where v=[v_(x)v_(x)]^(T) is said global motion vector, v_(x) is afirst vector component of said position-dependent global motion vectorin a first direction, v_(y) is a second vector component of saidposition-dependent global motion vector in a second perpendiculardirection, x=[xy]^(T) is an image element position in said frame. 10.The processor according to claim 9, wherein said calculator comprises: adifference calculator for calculating, for each image element in saidfirst group, a difference between said image element property value ofsaid image element and said image element property value of an imageelement having a corresponding image element position in said firstreference group; and a sum calculator for calculating said firstclassification parameter based on absolute values of said differences.11. The processor according to claim 9, wherein said calculator i)calculates, for each image element in said second reference group ofimage elements, a difference between said image element property valueof said image element and said image element property value of an imageelement having a corresponding image element position in said secondgroup, and ii) calculates a discard parameter based on absolute valuesof said differences.
 12. The processor according to claim 9, furthercomprising a classifier that classifies said first group as belonging tosaid background or said foreground based on said first classificationparameter and said second classification parameter.
 13. The processoraccording to claim 12, wherein said classifier classifies said firstgroup as belonging to said background or said foreground based on saidfirst classification parameter, said second classification parameter anda discard parameter.
 14. The processor according to claim 12, whereinsaid classifier classifies said first group as belonging to saidbackground or said foreground based on a comparison of said firstclassification parameter and a first threshold value a comparison ofsaid second classification parameter and a second threshold value. 15.The processor according to claim 14, wherein said classifier i)classifies said first group as belonging to said foreground if saidfirst classification parameter exceeds said first threshold value andsaid second classification parameter exceeds said second thresholdvalue, and ii) classifies said first group as belonging to saidbackground if said first classification parameter is smaller than saidfirst threshold value and said second classification parameter issmaller than said second threshold value.